Goto

Collaborating Authors

 garbage collection


DumpKV: Learning based lifetime aware garbage collection for key value separation in LSM-tree

Zhuang, Zhutao, Zeng, Xinqi, Chen, Zhiguang

arXiv.org Artificial Intelligence

Key\-value separation is used in LSM\-tree to stored large value in separate log files to reduce write amplification, but requires garbage collection to garbage collect invalid values. Existing garbage collection techniques in LSM\-tree typically adopt static parameter based garbage collection to garbage collect obsolete values which struggles to achieve low write amplification and it's challenging to find proper parameter for garbage collection triggering. In this work we introduce DumpKV, which introduces learning based lifetime aware garbage collection with dynamic lifetime adjustment to do efficient garbage collection to achieve lower write amplification. DumpKV manages large values using trained lightweight model with features suitable for various application based on past write access information of keys to give lifetime prediction for each individual key to enable efficient garbage collection. To reduce interference to write throughput DumpKV conducts feature collection during L0\-L1 compaction leveraging the fact that LSM\-tree is small under KV separation. Experimental results show that DumpKV achieves lower write amplification by 38\%\-73\% compared to existing key\-value separation garbage collection LSM\-tree stores with small feature storage overhead.


Tips for making the most of 64-bit architectures in langage design, libraries or garbage collection

Sonntag, Benoît, Colnet, Dominique

arXiv.org Artificial Intelligence

The 64-bit architectures that have become standard today offer unprecedented low-level programming possibilities. For the first time in the history of computing, the size of address registers far exceeded the physical capacity of their bus.After a brief reminder of the possibilities offered by the small size of addresses compared to the available 64 bits,we develop three concrete examples of how the vacant bits of these registers can be used.Among these examples, two of them concern the implementation of a library for a new statically typed programming language.Firstly, the implementation of multi-precision integers, with the aim of improving performance in terms of both calculation speed and RAM savings.The second example focuses on the library's handling of UTF-8 character strings.Here, the idea is to make indexing easier by ignoring the physical size of each UTF-8 characters.Finally, the third example is a possible enhancement of garbage collectors, in particular the mark \& sweep for the object marking phase.


Why Your Kubernetes Ship Is Sunk without Machine Learning - The New Stack

#artificialintelligence

With the rise of containerized services based on service-oriented architecture (SOA), the need for orchestration software like Kubernetes is rapidly increasing. Kubernetes is ideally suited for large-scale systems, but its complexity and lack of transparency can result in increased cloud costs, deployment delays and frustration among stakeholders. Used by large enterprises to scale their applications and underlying infrastructure vertically and horizontally to meet varied loads, the fine-grained control that makes Kubernetes so adaptable also makes it challenging to tune and optimize effectively. The Kubernetes architecture makes autonomous workload allocation decisions within a cluster. However, Kubernetes in itself doesn't ensure high availability. It will easily operate in a production environment with only one primary node.


GitHub - gorgonia/gorgonia: Gorgonia is a library that helps facilitate machine learning in Go.

#artificialintelligence

Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow. The primary goal for Gorgonia is to be a highly performant machine learning/graph computation-based library that can scale across multiple machines.


3 Reasons Why Python is So Popular With Developers in 2022?

#artificialintelligence

Whether you are brand new to programming or you have some experience with programming but decided to take on the challenge of learning a new language, you've got a lot of choices out there. There are a billion in one different programming languages that are going to have some use in the marketplace. Like any language, Python has its pros and its cons, but the simple fact of the matter is Python has become one of the most popular languages in the entire world. Python is a programming language that was designed to make it easy for programmers to write software, so developers can spend more time on their projects. The language is often used in the field of machine learning and artificial intelligence, where it is one of the most popular choices among developers.


Lisp machine - Wikipedia

#artificialintelligence

Lisp machines are general-purpose computers designed to efficiently run Lisp as their main software and programming language, usually via hardware support. They are an example of a high-level language computer architecture, and in a sense, they were the first commercial single-user workstations. Despite being modest in number (perhaps 7,000 units total as of 1988[1]), Lisp machines commercially pioneered many now-commonplace technologies, including effective garbage collection, laser printing, windowing systems, computer mice, high-resolution bit-mapped raster graphics, computer graphic rendering, and networking innovations such as Chaosnet.[citation The operating systems were written in Lisp Machine Lisp, Interlisp (Xerox), and later partly in Common Lisp. Artificial intelligence (AI) computer programs of the 1960s and 1970s intrinsically required what was then considered a huge amount of computer power, as measured in processor time and memory space.


7000 Free Pluralsight Courses to Build in-demand tech skills without leaving your house

#artificialintelligence

This course is designed to give you a solid foundation of the fundamentals of the Spring Framework. It covers how to get started as well as advanced configuration techniques with Spring using the most recent versions. A course covering the fundamentals of using Spring Security for securing Java applications built with Spring MVC. Spring Boot and Angular make a great team! See how all the pieces work together by building a full app, step by step.


Google Duplex: It was my son talking to me or Google Duplex? - My Empty Mind

#artificialintelligence

Google Duplex: It was my son talking to me or Google Duplex? It was my son talking to me or a Robot? Yes, that's what this new technology may cause you to think in future. D1: Capture bad records while loading csv in Spark Loading a csv file and capturing all the bad records is a very common task in ETL projects. The bad records are analyzed to take correctiv... New iPhone Release (Sept 2018) VS iPhone X: Features, Specification and Rumors With September coming closure discussions and rumors has put iPhone in the spotlight again. This autumn Apple Inc. may come up with thr... L3: Python or Scala for Apache Spark?


John Mccarthy

#artificialintelligence

John McCarthy, who died last week at the age of 84, was one of the true giants of computer science. Most remarkable about his contributions are their diversity, their depth, and how they span both theory and practice. To talk about him it is necessary first to dispel an unjustly negative connotation. McCarthy was one of the founders of the discipline of artificial intelligence, its most forceful advocate and the inventor of its very name. In the "AI Winter" episode of the late 1970s and 1980s, that name suffered some disrepute as a result of a scathing report by James Lighthill blaming AI researchers for over-promising.


Which Programming Language Is Best for Big Data?

#artificialintelligence

Nothing is quite so personal for programmers as what language they use. Why a data scientist, engineer, or application developer picks one over the other has as much to do with personal preference and their employers' IT culture as it does the qualities and characteristics of the language itself. But when it comes to big data, there are some definite patterns that emerge. The most important factor in choosing a programming language for a big data project is the goal at hand. If the organization is manipulating data, building analytics, and testing out machine learning models, they will probably choose a language that's best suited for that task.